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The contents of this document dcF.cribe the revised 
Center for Urban Education (CU.E.) plan for carrying out evaluation 
of the Satellite Academies Program, The 10 "area,^ of interest" are as 
follows: (1) Instructional and guidance staff si lection and training; 
(2) Program entrance and exit criteria; (3) Legal issues; (4) 
Guidance and pupil services; (5) Curriculum dev :-lopment ; (6) Cost 
analysis; (7) Program planning and development; (8) Selection of 
Satellite Academy sites; (9) Establishment and maintenance of 
school-industry liaison; (10) The initiation and development of plans 
for future expansion. The contents of this document include: (1) 
Evaluation Hodel; (2) Stages of Evaluation; (3) Evaluation Process 
Diagram; (4) Schedule of Tasks; (5) Categories of Investigation; (6) 
Major Evaluation Questions; (7) Statement of Expenditures to Date, 
(Author/JM) 
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The contents of this document describe the revised C.U.E. 
p^an for carrying c-)ut c^valuation ot the Satellite Academies Program 
as stipulated in AmenchnLint of Solicitation/Modification of Contract 
it 11, Project // 6-2868, June 15, 1971, and in subsequent correspon- 
dence of July 30, 1971. The ten (10) "areas of interest" agre^ed upon 
'vverc as follows: (1) Instructional and guidance staff selection and 
training; (2) Program entrance and exit criteria; (3) Legal issues 
(4) Guidance and pupil services; (5) Curriculum development; (6) 
Cost analysis; (7) Program planning and development; (8) Selec- 
tion of Satellite Academy sites; (9) Establishment and maintenance 
of school-industry liaison; (10) The initiation and development of 
plans f c r future expansion. 

The contents of this document include: (1) Evaluation Model; 
(2) Stages of Evaluation; (3) Evaluation Process Diagram; (4) 
Schedule of Tasks; (5) Categories of Investigation; (6) Major Eval- 
uation Questions; (7) Statement of Expenditures to date. 



SATELLITE ACADEMIES PEOGRAM 



EVALUATION MODEL 



INPUT 
(GOALS) 



ACTION^:^ 
(ACTIVITY) 



FEEDBACK ^_ 
'(CONTEOL)^^ 



OUTPUTS 
(EESULTS) 



U. S.O.E. 



' S. A. P. Evaluation is Process (Activity) Focused 



SATELIJTE ACADEMIES PROGi^AM EVALlJATi:^N STAGES 



STAGE I % o£ Project Cumulati^ e 

Completion 

Conceptualization and Planning 30 

!. Srudy the S, A. P. 3 3 

2. S'.udy the contract reouirements 3 6 

3. Tesign the project completion 

criteria 4 10 

4. Review the related !iterature 1 U 

5. Identify the evaluation model 2 13 

6. Identify the categories of 

investigation 2 15 

7. Identify the sub-categories 

of investigation 2 17 

&. Formulate the auestions 5 22 

9. Identify the tasks 4 26 

10. Determine the schedule 2 28 

11. Assign the personnel 2 30 



STAGE II 



Instrum.ent development 20 

1. Determine Measurement 

requirements I 31 

2. Design alternative instruments 4 35 

3. Analyze the alternative 

instruments 3 38 

4. Apply the cost criteria to 

each alternative 3 41 

5. Determine the benefit of each 

alternative 3 44 

6. Select the instruments 2 46 

7. Pilot the instruments 4 50 



Evaluation Stages (concluded) 
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STAGE III % of Project Cumulative % 

Completion 



ta 


collection 


40 




1, 


Determine, data requirements 


I 


51 


2, 


Select data collectors 


3 


54 


3, 


Train the data collectors 


4 


68 


4 


First data collection round 


10 


72 


5, 


Analyze data 


4 


74 


6 


Discuss the results in-house 


2 


84 


7 


Sec on 'ata c^jllection 


10 


88 


8 


Analy iata 


4 


90 


9. 


Discuss the results 


2 





STAGE IV 



E eporting 10 

1. Prepare the report draft 4 94 

2. Collegia I review 2 96 

3. Reportrevision 2 9C 

4. Submit the report 2 100 
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S. A.P. EVALUATiOxN PROCESS 



Concej}tiial ization 
and Planning 
(lA) 



(IB) 



ST AH T 



EVALUATE 
PROGRESS 

DECISION TO 
REORGANIZE 



CAREER ED. 
DIV. OR GAiNIZ ED 



(lA) 



STUDY 
S. A. P. 



STUDY CONTRACT 
REQUIREiMENTS 
si/ 

de:sign the 
project completion 
criteria 

STUDY 
RELATED LITERATURE 



IDENTIFY THE 
SUB-CATEC}ORIES OF 
INVESTIGATION 

FOR MU LA TE 

MAJOR 
QUESTIONS 

IDENTIFY 
PROJECT 
TASKS 

DETER MINE 

PROJECT 
SCHEDULE 



IDENTIFY THE 
EVALUATION MODEL 

i 

IDENTIFY THE 
CATEGORIES OF 
■ INVESTIGATION 



V 

ASSIGN 
PERSONNEL 

IS PLANNING 
OPTIMAL? 



V 

(IB) 



(2) 



Insti^ument 
Development 
(2) 

determine 
measure\u-:nt 
requirements 

J. 

DESIGN 
ALTER iNATIVE 
INSTRUMENTS 

4/ 

ANALYZE 
ALTERNATIVE 
INSTRUMENTS 

APPLY COST 
CRITERIA TO 
EA C H ALTER NA 11 VE 

DETERMINE 
COST/ BENEFIT 
OF 

EACH ALTERNATIVE 

SELECT 
INSTRUMENTS 

PILOT 
INSTRUMENTS 

ARE 
INSTRUMENTS 
OPTHMAL? 



(3A) 



ERIC 
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S.A.P. F,vahui.lit)i^ Process (concluded) 



Data Collection 



(3A) 

DETER MINE DATA 
REQUIEEAIENTS 



(3B) 



X 



SEEECT DATA 
COLLECTORS 

TRAIN DATA 
COLLECTORS 

FIRST ROUND 

DATA 
COLLECTION 

I 

ANALYZE 
FIRST ROUND 
DATA 

DISCUSS 
RESULTS 

ARE RESULTS 
SATISFACTORY? 

INTERIM 
REPORT 

i 

■ SECOND ROUND 
DATA 
COLLECTION 



ANALYZE 
DATA 
si, 

DIS SS 
RESULTS 

4^ 
ARE 
RESULTS 
SATISFACTORY? 



(3B) 



V 
(4) 



(4) 



PREPARE 
DRAFT 
REPORT 

DISCUSS 
REPOR T 

R E VISE 
REPORT 

i 

SUBMIT 
REPOR T 



i 



PROJECT 
COMPLETE 




TASK IDENTIFICATION 

0-1 Evaluate progress 
0-2 



0- 
0. 

0- 
0- 



Decision to reorganize 
Career Ed. Div. organized 
Is plan optimal? 
Axe instruments optimal? 
Are results optimal? 



I-l Study the S. A. P. 

1-2 Study the contract requirements 

1-3 Design the project completion 

criteria evaluation 

1-4 Review the related literature 

1-5 Identify the model 

1-6 Identify categories of investigation 





SUMMARY NETWORK 





1-7 Identify the sub -categories of II-l 

investigation 

1-8 Formulate the questions II-2 

1-9 Identify the tasks II- 3 
I-IO Detcrz~t*iine 1 schedule 

I-ll Assign the personnel II-4 



Determi 
r cqui re^ 
Des ign 
Analyze] 
instruxnl 
Apply tl 
each alii 



NETWORK FOR S^A.P, EVALUATION PLAN 




L Dtt(^rminc measurement 
requirement 
II-2 Design alternative instrument 
II -3 An.ily/:e the alternative 

instruments 
II-4 Apply the cost criteria to 
each a ItLtr native 



II-5 Deteriinine the b.encfit of each 

alternative 
11-6 Select the instruments 
II- 7 Pilot the instruments 



III-l Dete 

III-2 Sele< 

III -3 Trai- 

ni-4 1st d 

in- 5 Anar 



V- 




III-l Doturir.inc data requirements III-6 Discuss the results in hours IV-I Prepar 

III-2 Select data collectors 111-7 2nd data collection IV-2 E eview 

III-3 Train data collectors ' III-8 Analyze data IV-3 Report 

Ul-t l5t data collection round III-9 Discuss the results IV-4 Submit 

III -5 Analyze data 



r equi r c ments 
ectors 
-ctors 
on round 



III- 6 

III- 7 
J II- 8 
III -9 



Discuss the results in hours 
2nd data collection 
Analyze data 
Disci' 5 s th(j results 



IV-1 Prepare the report draft 

IV-2 Review the report 

IV-3 Report revision 

IV-4 Subnnit the report 
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CATEGCRIES OF INVESTIGATION 
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1. Program Management 

a. Program Planning and Development 

b. Establishment and Maintenance of School-Employer Liaison 

c. Planning 

d. Cost Analysis 

e. Site Selection 
X. Legal Issues 



2. Instructional Personnel 

a. Instructional Staff R ecruitment, Selection, Training and 
Evaluation 

b. Legal Issues 



3. Pupil Personnel Services 

a. Guidance and Counselling 

b. Personal Welfare 

c. Payment 

d. Legal Issues 



4. Pupil Personnel 

a. Program Entrance and Exit Criteria 

b. Population Characteristics 

c. Attendance and Achievement (Work) 

d. Attitudes (Pupil and Parent) 

e. Pupil Behavior 

f. Legal Issues 



5. Curriculum 

a. Scope, Sequence and Instructional Strategies 

b. Organization 

c. Relationship v^ith Actual Job Performance Standards 

d. Performance Objectives and Evaluation 

e. Legal Issues 
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PPOGKAM MANAGEMENT 
L What are the S. A. P. goals? 

2. Why were these goals chosen? 

3. What are S. A. P. objectives? 

4. Why were these objectives chosen? 

5. What were the progress indicators chosen for S. A. P, ? 

6. Why were these indicators chosen? 

7. What was the time frame of S, A, P, objectives? 

8. How was this time frame chosen? 

9. What minimum levels of progress toward objectives were 
established? 

10. Why were these levels chosen? 

11. What program tasks were identified to achieve S, A.P. objectives? 

12. What alternative tasks were identified? 

13. What was the planned completion schedule for S.A,P. ? 

14. What has been the actual schedule of objective completion? 
•5. V/hat were the management personnel requirements? 

16. What were the criteria for management personnel selection? 

17. How were management personnel selection? 

18. What procedures were planned for evaluation of management 
personnel? 

19. What is the plan to expand S.A.P, program in the future? 



Program Management (concluded) 

39. How are these facility costs estimated? 

40. What are the estimated materials costs for installation of 
another site in future expansion? 

41. How are these materials costs estimated? 

42. What are the estimated training costs for installation of 
another site in future expansion? 

43. How are these training costs estimated? 

44. What are the estimated staffing costs for installation of 
another site in future expansion? 

45. How are these staffing costs estimated? 

46. What are the estimated costs for equipment replacement in 
future expansion? 

47. How are these equipment replacement costs estimated? 

48. What are the estim.ated costs for facilities replacement in 
future expansion? 

49. How are the costs for facilities replacement estimated? 

50. What are the estimated costs of maintenance of equipment in 
future expansion? 

51. How are these equipment maintenance costs estimated? 

52. What are the estimated costs of maintenance of facilities in 
future expansion? 

53. How are these facilities maintenance costs estimated? 

54. What are the estimated staffing costs to operate these sites 
in future expansion? 

55. How are these staff costs estimated? 



Program Managment (continued) 

20. How were the S. A. P, school- employer relations established? 

Zl. How have school-employer relations been maintained? 

22. What were the criteria for selecting employers lor S, A, P. ? 

23. What legal requirements xor employer selection were identified? 

24. What are the S.A.P, reouirements for school-employer liaison? 

25. What legal constraints of school-employer liaison were identified? 

26. What are the current problems in s choo I- employer liaison? 

27. What are the foreseen constraints in the future of school- 
employer relations? 

28. How might problems in school- employer relations be avoided? 

29. What were the criteria for selecting S, A. P. sites? 

30. How many alternative J Ites were considered? 

31. What legal or contractual constraints foi site selection were 
identified ? 

32. What are the characteristics of the current S, A. sites? 

33. What current requiremenLo for site selection were identified? 

34. What were the costs per - par ticipant (pupil) for S, A, P. ? 

35. How were these costs determined? 

36. How do these costs per participant (or costs per pupil) com- 
pare with other training program?5 or vocation educational 
programs ? 

37. What are the estimated pre-adoption costs for installation of 
another 

3C. What are the estimated facility costs for installation of 
another site in future expansion? 
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INSTRUCTIONAL PERSONNEL 

L What niuTiber of instructional staff was deemed necessary for 
the implementation of S. A. P. , i. e. , academic, skill training, 
etc. ? 

2. What were the conceptualized characteristics o£ the instruc- 
tional staff? 

3. What were the actual characteristics of the selected staff? 

4. What were the conceptualized characteristics of auxilliary 
instruction personnel? 

5. What were the actual characteristics of selected auxilliary 
instructional personnel? 

6. What was the rationale for the different instructional staff 
selection standards? 

7. What was the talent pool from which instructional personnel 
were to be drawn? 

8. What methods of instructional staff recruitment were employed? 

9. What was the number of applicants responding to recruitment 
efforts ? 

10. Which recruitment method yielded the greatest number of res- 
pondents ? 

IL Which souce yielded the largest number of qualified respondents? 

12. What was the mechanism for staff selection? 

13. How did actual instructional staff selection deviate from plan? 

14. How might selection problems be avoided? 

15. What was the rationale for the decision to train instructional 
staff? ' 

16. What was the nature of instructional staff training? 



Instructional Personnel (concluded) 

17, What organizational structures were planned to accomplish 
instructional staff training? 

lb. How did instructional staff training deviate from plan? 

19. How might training problems be avoided? 

20. What is the record cf retention of instructional personnel? 

21. What procedures were planned for evaluation of instructional 
staff performance? 

22. What has been the performance record of instructional 
personnel? 

23. What legal constraints were identified as bearing on the 
employment of instructional personnel? 

24. What procedures were planned in consideration of thes 

c onstraints ? 

25. What legal constraints subsequently arose? 

26. What steps were taken in consideration of these additic 
constraints? 
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PUPIL PERSONNEL SERVICES 

1, What guidance/counselling services were planned? 

2, Why were the planned guidance /counse lling services selected? 

3, What guidance/counselling services have actually been provided? 

4, How have guidance/counselling services been evaluated? 

5, What has been the results of the evaluation of guidance/coun- 
selling services? 

6, What were anticipated personnel requirements for guidance/ 
counse lling ? 

7, How were guidance/counselling personnel recruited? 

8, How were guidance/counselling personnel selected? 

9, How were guidance/c ouniseUing personnel trained? 

10, How were guidance/counselling personnel evaluated? 

IL What problems have been encountered in the guidance/ 
counselling program? 

12. How might guidance/counse lling problems be avoided? 

13. What pupil personal welfare services were planned? 

14. Why were these pupil person?* elfare services selected? 

15. What actual pupil personal welfare services have been 
provided? 

16. What were the anticipated staff requirements for pupil 
personal welfare services? 

17. What were the actual staff recuirements for pupil personal 
welfare services? 

18. What procedures for evaluation of personal welfare services 
were planned? 



Pupil Personnel Services (concluded) -16- 

19. What has been the evaluation of personal welfare services? 

20. What provisions for payment to pupils was plannea? 
2L What is the rationale for payment to pupils? 

22. What problems h^ve been encountered in providing payments 
to pupils ? 

23. How might problems due to pupil payment be avoided? 

24. What legal issues have been encountered in providing pay- 
ment to pupils ? 

25. How have these legal issues been resolved? 
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PUPIL PERSONNEL 



U What were the characteristics of the projcctorl pupil population 
of the S.A.P. program? 

2. What was the rationale for the selection of those characteristics? 

3. Did the characteristics of the actual pupil popul^^tion to those of 
the projected pupil population? 

4. If significant variation between projected and actual pupil popula- 
tion developed, what were the factors which influenced deviation? 

5. What considerations influenced the determination of pupil popula- 
tion size? 

6. Did the actual pupil population size conform to the projected pop- 
ulation size? 

7. If significant variation developed between projected and actual 
population size, what factors influenced deviation? 

8. What was the projected population phase-in schedule and the 
rationale lor its determination? 

9. If the projecx-ed phase-in schedule varied significantly from the 
actual events, what factors influenced deviation? 

10. How did the attendance pattern of the S. A. P. pupil populaVion 
compare with attendance norms for similar populations in 
traditional school settings? 

11. What factors can be associated with comparitive attendance 
pattern results? 

12 What was the pupil retention record of the S, A. P, program? 

13. How does the S.A.P. pupil retention record compare with that 
of similar populations in traditional settings? 

14, What factors can be associated with comparative retention 
rate results r 



Pupil Personnel (concluded) 
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lb. vVhat xcrmal and/ur informal systems were employed to 

measure job skill acquisition levels?. 

16. What were the rationales for decisions to utilize selected 
systems of job skill acquisition measures? 

17. Wha.t was the achievement record of S. A. P. pupils in job 
skill acquisition? 

18. What was the nature and the contexts of pupil behavior prob- 
lems observed in the course of S. A. P. operations? 

19. Which pupil behavior problems could be clearly associated 
with the special circumstances of S. A. P. operations? 

20. What mechanisms, techniques and strategies were success- 
fully and unsuccessfully employed in response to pupil behavior 
problems ? 

21. What were pupil attitudes towards various aspects of the S. A. P. 

22. What were parent attitudes towards various aspects of the £. A. P 

23. What was the nature of legal problems encountered prior to and 
riuring S. A.P, operations in regard to pupil personnel? 

ii ^v-. V. cr..- ;:)upil pursonnel legal problems managed and solved 
pn ":r.'l luring S.A.P. operations? 

i:-^. ■■.V::!<.;i , . -s:.;nnei legal problem.s were particularly difficult 

r. u iTi n ci L- ."1 1 r ! ^ o I v e ? 

26. What solutions were developed for legal problems of special 
difficulty? 

27. How did pupil personnel legal problems effect program planning 
and operation? 
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CURRICULUM 



1, What is the scope and seouence and instructional strategies 
of the " Vestibuie"curricuium for each Academy? 

2, VVhat was the rationale for developing "Vestibule" curricula? 

3, What is the scope and seouence and instructional strategies 
of the "Academic" curriculuxn for each Academy? 

4, What is the scope and seouence and instructional strategies 
of the "Electives" curriculum for each Academy? 

5, Are the subjects in the "Academic" and "Elective" curricula 
w or r e tated ? 

6, W! . types of professional personnel were involved in curricula 
contj truction and to what extent? 

7, Why were these types selected? 

S. What types of non- profes sional personnel were involved in 
curricula construction and to what extent? 

9, Why were these types selected? 

10. Are programmed materials employed in curricula application? 

11. Which instructional materials were developed by S. A. P. per- 
sonnel? 

12. What are the student options in the "Academic" curricula? 

13. Eo combined curricula meet high school diploma require- 
ments at state and local levels? 

14. What was the rationale for the selection of only two occupa- 
tional areas? 

15. What was the rationale for the selection of the clerical and 
health areas of study? 



